Explaining Anomalies using Denoising Autoencoders for Financial Tabular Data

Recent advances in Explainable AI (XAI) increased the demand for deployment of safe and interpretable AI models in various industry sectors. Despite the latest success of deep neural networks in a variety of domains, understanding the decision-making process of such complex models still remains a challenging task for domain experts. Especially in the financial domain, merely pointing to an anomaly composed of often hundreds of mixed type columns, has limited value for experts. Hence, in this paper, we propose a framework for explaining anomalies using denoising autoencoders designed for mixed type tabular data. We specifically focus our technique on anomalies that are erroneous observations. This is achieved by localizing individual sample columns (cells) with potential errors and assigning corresponding confidence scores. In addition, the model provides the expected cell value estimates to fix the errors. We evaluate our approach based on three standard public tabular datasets (Credit Default, Adult, IEEE Fraud) and one proprietary dataset (Holdings). We find that denoising autoencoders applied to this task already outperform other approaches in the cell error detection rates as well as in the expected value rates. Additionally, we analyze how a specialized loss designed for cell error detection can further improve these metrics. Our framework is designed for a domain expert to understand abnormal characteristics of an anomaly, as well as to improve in-house data quality management processes.Comment: 10 pages, 4 figures, 3 tables, preprint versio

FinDiff: Diffusion Models for Financial Tabular Data Generation

The sharing of microdata, such as fund holdings and derivative instruments, by regulatory institutions presents a unique challenge due to strict data confidentiality and privacy regulations. These challenges often hinder the ability of both academics and practitioners to conduct collaborative research effectively. The emergence of generative models, particularly diffusion models, capable of synthesizing data mimicking the underlying distributions of real-world data presents a compelling solution. This work introduces 'FinDiff', a diffusion model designed to generate real-world financial tabular data for a variety of regulatory downstream tasks, for example economic scenario modeling, stress tests, and fraud detection. The model uses embedding encodings to model mixed modality financial data, comprising both categorical and numeric attributes. The performance of FinDiff in generating synthetic tabular financial data is evaluated against state-of-the-art baseline models using three real-world financial datasets (including two publicly available datasets and one proprietary dataset). Empirical results demonstrate that FinDiff excels in generating synthetic tabular financial data with high fidelity, privacy, and utility.Comment: 9 pages, 5 figures, 3 tables, preprint version, currently under revie

FedTabDiff: Federated Learning of Diffusion Probabilistic Models for Synthetic Mixed-Type Tabular Data Generation

Realistic synthetic tabular data generation encounters significant challenges in preserving privacy, especially when dealing with sensitive information in domains like finance and healthcare. In this paper, we introduce \textit{Federated Tabular Diffusion} (FedTabDiff) for generating high-fidelity mixed-type tabular data without centralized access to the original tabular datasets. Leveraging the strengths of \textit{Denoising Diffusion Probabilistic Models} (DDPMs), our approach addresses the inherent complexities in tabular data, such as mixed attribute types and implicit relationships. More critically, FedTabDiff realizes a decentralized learning scheme that permits multiple entities to collaboratively train a generative model while respecting data privacy and locality. We extend DDPMs into the federated setting for tabular data generation, which includes a synchronous update scheme and weighted averaging for effective model aggregation. Experimental evaluations on real-world financial and medical datasets attest to the framework's capability to produce synthetic data that maintains high fidelity, utility, privacy, and coverage.Comment: 9 pages, 2 figures, 2 tables, preprint version, currently under revie

RESHAPE: Explaining Accounting Anomalies in Financial Statement Audits by enhancing SHapley Additive exPlanations

Detecting accounting anomalies is a recurrent challenge in financial statement audits. Recently, novel methods derived from Deep-Learning (DL) have been proposed to audit the large volumes of a statement's underlying accounting records. However, due to their vast number of parameters, such models exhibit the drawback of being inherently opaque. At the same time, the concealing of a model's inner workings often hinders its real-world application. This observation holds particularly true in financial audits since auditors must reasonably explain and justify their audit decisions. Nowadays, various Explainable AI (XAI) techniques have been proposed to address this challenge, e.g., SHapley Additive exPlanations (SHAP). However, in unsupervised DL as often applied in financial audits, these methods explain the model output at the level of encoded variables. As a result, the explanations of Autoencoder Neural Networks (AENNs) are often hard to comprehend by human auditors. To mitigate this drawback, we propose (RESHAPE), which explains the model output on an aggregated attribute-level. In addition, we introduce an evaluation framework to compare the versatility of XAI methods in auditing. Our experimental results show empirical evidence that RESHAPE results in versatile explanations compared to state-of-the-art baselines. We envision such attribute-level explanations as a necessary next step in the adoption of unsupervised DL techniques in financial auditing.Comment: 9 pages, 4 figures, 5 tables, preprint version, currently under revie

SPIRITUAL AND MORAL EDUCATION OF THE GROWING GENERATION

The topic of the spiritual and cultural formation of the younger generation at all times has been the subject of close comprehension of philosophers, religious scholars, teachers, representatives of many ethnic groups who study it. Each ethnic group developed certain methods of education (training and upbringing) of young people, the formation of its spiritual and cultural appearance. Different cultures: eastern and western, have developed their models, standards of education, spiritual and cultural development, and upbringing, in which both universal and specific ethnonational aspects are present. The article considers the problem of spiritual and moral education of the younger generation of the Republic of Uzbekistan, aimed at reviving national values, improving the system of national education, and educating a harmoniously developed generation in the spirit of patriotism and love for the Motherland. It also reveals the importance of studying the history of Uzbekistan in educating young people in the spirit of the ideology of national independence. The cultural past does not disappear, it persists for generations, works for the present, laying the foundation for the future. The culture of the past is always necessary for modernity, which is experiencing a crisis of spirituality and morality. The present must be compared with the past to select a new trend in cultural development. Turning to the past cultural, intellectual, values allows us to understand the present, find ways to overcome the crisis in culture, associated not only with a drop in the quality of education but also with the spiritual and cultural education of young people, as well as to bridge the gap between the sides of a single cultural and intellectual process. The article examines the ideas of educating the younger generation in the works of such oriental thinkers and educators like Abu Raikhan Beruni, al-Farabi, Ibn Sina, analyzes their philosophical views on education and such vital tasks as finding the meaning of life, researching good and evil, defining the concepts of justice, compassion, etc. Attention is paid to the relevance of studying political and legal doctrines, works of thinkers of the East, which have an important role in educating the younger generation in the spirit of patriotism and high legal culture. The article raises the problems of educating modern youth in the spirit of patriotism based on national traditions